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Two  lower  estimates  in  greedy  approximation1 


V.N.Temlyakov 

University  of  South  Carolina,  Columbia,  SC  29208,  USA 

Abstract.  We  prove  one  lower  estimate  for  the  rate  of  convergence  of  Pure  Greedy 
Algorithm  with  regard  to  a  general  dictionary  and  another  lower  estimate  for  the  rate 
of  convergence  of  Weak  Greedy  Algorithm  with  a  special  weakness  sequence  t  =  {t}, 
0  <  t  <  1,  with  regard  to  a  general  dictionary.  The  second  lower  estimate  combined 
with  the  known  upper  estimate  gives  the  right  (in  the  sense  of  order)  dependence  of 
the  exponent  in  the  rate  of  convergence  on  the  parameter  t  when  t  — >•  0. 


1.  Introduction 

This  paper  is  a  followup  to  the  paper  [L]  of  E.  Livshitz.  In  Sections  2,3  we  study 
the  convergence  rate  of  Pure  Greedy  Algorithm  and  in  Section  4  we  study  Weak 
Greedy  Algorithm.  We  define  first  the  Pure  Greedy  Algorithm  (PGA)  in  Hilbert 
space  H.  We  describe  this  algorithm  for  a  general  dictionary  V.  If  /  G  H,  we 
let  g(f )  G  V  be  an  element  from  V  which  maximizes  |(/, g)\.  We  shall  assume  for 
simplicity  that  such  a  maximizer  exists;  if  not  suitable  modifications  are  necessary 
(see  Weak  Greedy  Algorithm  below)  in  the  algorithm  that  follows.  We  define 

G(f,V)  :=  (f,g(f))g(f) 

and 

R(f,V)  :=f-G(f,V). 

Pure  Greedy  Algorithm  (PGA).  We  define  Ro(f,V)  :=  /  and  Go(f,V)  :=  0. 
Then,  for  each  m  >1,  we  inductively  define 

Gm(f,  V)  :=  V)  +  G(Rm-i(f,  V),V) 

Rm(f,V )  :=f-Gm(f,V)=R(Rm-1(f,V),V). 

For  a  general  dictionary  V  we  define  the  class  of  functions 

A°(V,  M)  :=  {f  e  H  :  f  =  ckwk ,  wk  G  V,  #A  <  oo  and  ^  \ck\  <  M} 

fee A  k£ A 

and  we  define  AifiD,  M)  as  the  closure  (in  H)  of  A°(T>,  M).  Furthermore,  we  define 
Ai{V)  as  the  union  of  the  classes  Ai(T>,M)  over  all  M  >  0.  For  /  G  A\(V),  we 
define  the  “semi-norm” 

l/Ui  (V) 

as  the  smallest  M  such  that  /  G  A\(V,M). 

1This  research  was  supported  by  the  National  Science  Foundation  Grant  DMS  9970326  and 
by  ONR  Grant  N00014-91-J1343 
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It  was  proved  in  [DT]  that  for  a  general  dictionary  D  the  Pure  Greedy  Algorithm 
provides  the  following  estimate 

(1.1)  ||/-Gm(/,X>)||  <  |/Ul(p|m-1/6. 

(In  this  and  similar  estimates  we  consider  that  the  inequality  holds  for  all  possible 
choices  of  {Gm}.)  The  paper  [DT]  contains  also  an  example  of  a  dictionary  V  and 
an  element  /  such  that 

(1.2)  ll/-Gm(/,©)||>l|/U,(i,)m-1l2,  m>  4. 

We  proved  in  [KT]  a  new  estimate 

(1.3)  11/  -  G„(/,D)||  <  4| /|A(I))m-11/62 

which  improves  a  little  the  original  one  (see  (1.1)). 

E.  Livshitz  [L]  proved  that  there  exist  h  >  0,  a  dictionary  V  and  an  element 
f  &  H,  f  ^  0,  such  that 

||/-Gm(/,B)||>Cro-1G+i|/U,(B) 

with  a  positive  constant  C.  We  develop  and  refine  ideas  from  [L]  here  to  prove  the 
following  lower  estimate. 

Theorem  1.1.  There  exist  a  dictionary  V  and  an  element  f  e  H ,  f  A  0,  such 
that 

II/  — Gm(/,©)||  >Cm-l^\f\Aliv) 
with  a  positive  constant  C. 

In  Section  4  we  study  the  Weak  Greedy  Algorithm.  Let  a  sequence  r  =  {tk}kLi, 
0  <  tk  <  1,  be  given.  Following  [T]  we  define  Weak  Greedy  Algorithm  as  follows. 

Weak  Greedy  Algorithm  (WGA).  We  define  :=  f .  Then  for  each  m  >  1, 
we  inductively  define: 


!)■ 

pTm  E  is  any  satisfying 

>  tm  sup  \(ffn,_1,g}\ 

g£V 

2). 

. _  1?T 

•i  m  ’  J  m 

3). 

m 

GTm(f,V)  : 

3  = 1 


In  Section  4  we  discuss  the  following  question.  How  does  the  weakness  sequence 
r  affect  rate  of  convergence  of  WGA?  We  consider  here  only  the  special  case  of 
weakness  sequences  r  =  with  tk  =  t,  k  =  1,  2, . . . ,  0  <  t  <  1.  In  order 

to  stress  this  we  replace  in  notations  r  by  t.  It  is  known  from  [J]  that  WGA  with 
the  above  special  weakness  sequence  {t}  converges  for  any  0  <  t  <  1.  We  show  in 
Section  4  that  the  weakness  parameter  t  affects  the  rate  of  convergence  of  WGA  on 
the  class  AifiD).  For  the  WGA  we  have  the  following  upper  estimate  [T], 
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Theorem  1.2.  Let  V  be  an  arbitrary  dictionary  in  H .  Assume  r  :=  {tk}kLi  is  a 
nonincreasing  sequence.  Then  for  f  G  Ai(T>,M)  we  have 

m 

(1.4)  ||/-G^(/,C)||  <M(1  +  £i!)-'~/2<2+'~> 

k=l 


for  any  realization  of  GTm(f  ,V) . 

In  a  particular  case  r  =  {t},  (tk  =  t,  k  =  1,2,... ),  (1.4)  gives 

(1.5)  ||/  -  Gt(/,2>)||  <  M{1  +  mt2)-4/(4+2t),  0  <  t  <  1. 

This  estimate  implies  the  following  inequality 

(1.6)  ||/  -  G*ra(/,2?)||  <  C'i(f)m-“t|/U1(©)>  a  <  1/6, 

with  the  exponent  at  approaching  0  linearly  in  t.  We  prove  in  Section  4  that  this 
exponent  can  not  decrease  to  0  slower  than  linearly. 

Theorem  1.3.  There  exists  an  absolute  constant  b  >  0  such  that  for  any  t  >  0  we 
can  find  a  dictionary  Vt  and  a  function  ft  G  Ai{Vt)  such  that  for  some  realization 
Gtm{fti'E>t)  of  Weak  Greedy  Algorithm  we  have 

(1-7)  liminf  \\ft  -  Gtm(ft,Vt)\\mbt /\ft\Al(Vt)  >  0. 

ra— >-  oo 


2.  General  formulas 

We  will  be  constructing  simultaneously  two  sequences  of  elements  {xn}ff=N  and 
{9n}n=N-  A  number  N  will  be  chosen  later  to  be  large  enough.  Let  {ejl/Tj  be  an 
orthonormal  basis  for  H.  We  let  {xn}  have  the  following  form 


(2.1) 


n 

Xn  —  ^  '  Cln^kOki  XI  —  IV,  .  .  .  , 

k= 1 


and  the  {gn}  have  the  form 

(2.2)  9n+ 1  —  Tn+lXn  T  hn- |_i  T  ^n+ien+l,  xi  —  iV, . . . , 

with  the  sequences  {y^},  {hk},  {/&},  0  <  £&  <  1,  to  be  specified.  The  element  [jn 
will  be  also  specified  later.  We  always  assume  that 

(2.3)  ||^||  =  1,  k  =  N, . . . . 

We  complete  the  inductive  definition  of  sequences  {xn}ifLN  and  {^nK/Liv+i  by 
setting 


(2.4) 


Xn+l  ■ —  Xn 


{Xni  9n+l) 9n+l- 
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We  will  not  specify  the  sequences  {7^},  {hk},  {£&},  0  <  £&  <  1,  in  this  section  and 
get  here  some  general  formulas  under  assumption  that  the  above  sequences  satisfy 
some  conditions.  It  will  be  convenient  for  us  to  introduce  one  more  sequence 

(2.5)  q«,+i  . —  ( xn ,  gn 1 ) )  xi  —  .... 

We  assume  that  hn  +1  e  Spanjei, . . . ,  enj  for  all  n.  This  assumption  and  (2.4) 
imply 

^n+l,n+l  —  Qn+lCn+1- 

We  assume  that  the  sequences  {7^}  and  {<?&}  satisfy  the  following  condition 

(2.6)  1  -  Jn+lQn+l  =  n  =  N  +  1,.... 

Qn 

Let  us  list  some  identities  which  will  be  useful  later  on.  Using  (2.2)  and  (2.5)  we 
get  from  (2.4)  that 

(2.7)  £n_|_i  —  Xn  qn+l9n+l  —  (1  7«.+  l<7n+l)*£n  ^n+l^n+l  Qn+lCn+l^n+1  — 

Qn-\-l  7 

Xn  Qn+l^n+l  Qn+lSn+lTn+l  • 

Qn 

The  relations  (2.1)  and  (2.2)  imply 

(2-8)  Qn+ 1  —  (^m  9n+ 1 )  —  7n+l  II Xn  1 1  T  (xn,  ^n+1 )  • 

We  will  need  formulas  for  ( xn,gk )■  It  is  clear  from  (2.4)  that 

(2.9)  (xn,  gn)  =  0,  n  =  N+  1, - 

I.  Case  N  <  k  <  n.  By  (2.6)  and  (2.7)  we  get  for  n  >  N  +  2 

(2.10)  (. xn,gk )  =  (-^-Xn-!  -  qnhn,gk)  =  -^—{xn-U  gk)  ~  qn{hn,  gk). 

Qn  —  1  Qn  —  1 

Repeating  (2.10)  and  using  (2.9)  we  obtain  for  k  >  N  +  1 

n  n 

(2.11)  (xn,gk)  =  —(xk,gk)  ~  Qn  ( hi,gk)  =  -qn  (h^9k)- 

Q k  l=k+ 1  l=k+ 1 

Let  us  choose  now.  We  specify  /ijv+i  =  0  and  take  0  <  e  <  1.  Set 

(2.12)  gN  :=  eXN+i\\xN+i\\  2  +  Cqn+i 
with  £  such  that  ||(pv||  =  1.  Then 

(xn+i,9n)  =  e, 

and  xn+i  €  Span(^jv, gN+i)-  By  (2.10)  similarly  to  (2.11)  we  get  for  n  >  N  +  2 


(2.13) 


(xn,gN)=  qn  (xN+i,gN)-qn  T  ( hi,gN )■ 

qN+ 1  W2 
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II.  Case  k>n  +  l>N  +  2.  We  have  by  (2.2)  for  k  >  n  +  1 

(2.14)  9k+ 1)  —  {xn,  'Yk+lXk  “f“  ^fc+l)  —  7fc+  l{Xnt  Xk)  ~b  {xn,  ^fc+l)- 
Next, 

(2.15)  (xn,  xk)  =  ( xn ,  Xk-i  ~  qk.gk)  =  (x„,  xk-i)  ~  qk{xn,  9k )■ 

We  also  have 

(2.16)  (xni  9k)  —  (Xny  'YkXk  —  l  4“  hk)  —  'YkiXm  Xk—l)  A  (%m  hi fc) 

and 

(2.17)  {xn,xk_ i)  =  7fc1((a:n,5ffc)  -  (xn,hk)). 

Combining  (2.15)  and  (2.17)  we  get  from  (2.14) 

(2.18)  (xn,gk+ 1)  =  {.Ik1  ~  qk)lk+i(xn,9k)  +  (xn,hk+1  -  ^-!-hk). 

7  k 

Using  (2.6)  we  rewrite  (2.18) 

(2.19)  {xn,gk+ 1)  =  ——  {xm  9k)  +  {xn,hk+1  -  -^-hk). 

7  k  qk  1  7  k 

We  note  that  (2.4)  implies 

ll^nH  —  H^n+lH  A  {xn^9n-\- 1) 

and 

(2.20)  q2n+1  =  \\xn\\2 -\\xn+1\\2,  n  =  N, . . . . 

3.  Specifications 

Our  goal  is  to  prove  that  the  procedure  described  by  (2.4)  is  a  Pure  Greedy 
Algorithm  with  regard  to  the  dictionary  D  =  {gm}m=N  with  approriately  chosen 
N  and  /.  We  will  choose 

N 

xn  :=  — 1W1/2 1 1 x n 1 1  ^2  e*’ 

i=  1 

with  the  sequence  {||ccri||}  specified  below  and  we  define  gN  by  (2.12).  With  these 
two  starting  elements  we  use  the  procedures  (2.2)  and  (2.4)  to  get  the  sequences 
n  and  {9n}^=N-  We  set  /  :=  xn+i  and  prove  that  for  big  enough  N  (2.4) 
is  a  realization  of  PGA.  This  means  that  we  should  check  that  for  any  n  >  N  A  1 
and  any  m  >  IV,  m  /  n  +  1,  we  have 

(3.1)  l(  Qm  )|  <  qn+ 1. 

We  remind  that  by  the  definition  of  {qk}  we  have 

{Xni  9ri-\- 1)  —  qn-\- 1- 

We  will  prove  (3.1),  considering  separately  two  cases  m  >  n  +  1  and  m  <  n  A  1. 
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3.1  Numerical  sequences.  We  define  sequences  {HznlD^j,  {qn}n 12>  {7«.}5^=3 
depending  on  a  parameter  0  <  /3  <  1/4.  In  parallel  we  give  in  square  brackets 
asymptotic  estimates  for  j3  =  1/6.  This  will  give  a  construction  for  Theorem  1.1. 
We  set 

(3.2)  |M|  =  1;  \\xn\\2  =  (l-(3)2(l-2f3)-1n-1+2/3,  [§W2/3],  n  =  2,3,.... 

Then  by  (2.20)  we  have 

ql  =  ll^n-ill2  -  \\xn\\2  ~  (1  -  /3)2n~2+2/3,  n  =  2,3, ... , 

and 

(3.3)  qn  &  (1  -  I3)n-1+P ,  [^n-5''6]. 

From  (2.6)  we  get 

(3.4)  7n  =  On1  -  Qn-l  ~  Tl~P ,  [n~1/6]. 

We  will  also  need  the  following  relations 

(3.5)  7n+i||zJ2  -qn+1  ~  13(1  -  I3){1  -  2f3y1n~1+l3,  [- ^n~5/6]. 


(3.6)  ^±i  _  ~  (i  _  2f3)n-\  [In-1]. 

Tn  Qn  3 

(3.7)  Jn+i_Qn_  =  1  _  (1  +  0(i))n-i_ 

7  ra  Qn  —  1 

We  will  use  the  following  numerical  estimate  for  (3  =  1/6 

(3.8)  (1  -  /3)(1  -  2/3)~1/2  =  5(24)^1/2  <  1.021. 

3.2  The  sequence  {hn}.  We  already  agreed  to  set  Jin+i  =  0.  Let  for  n  >  N  +  2 
define  hn  as  follows 

OL 

(3.9)  hn  :=  — ^-$n,  0  <  an  <  a0  <  1/2, 

n  —  1 

with 

4* n  • —  'y  ' 

k 

where  0n(f)  :=  X(an,n-i)(t)  is  the  characteristic  function  of  (an,  n  —  1)  with  a 
parameter  0  <  a  <  1.  In  the  following  numerical  estimates  we  set  a  =  0.05.  The 
sequence  {«„},  0  <  an  <  ckq  <  1/2,  will  be  chosen  later.  Then  we  have 


(3.10) 


4>n||  Rj  (1  —  a)1/2/!1/2,  [0 .951/2n1/2]; 
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and 

(3.11)  ||$n||i  :=  ^  ~  (1  ~  a)n ,  [0.95n]. 

k 

It  is  clear  from  (2.2),  (2.3)  and  definition  of  that  — *  1  as  n  — *  oo.  This  and 

(2.7)  imply  that  for  big  n  we  will  have 

(3.12)  an,fe  <  -(1  +  o(l))<7n,  n,  k>Nu 
and 

(3.13)  Am  :=  |(xm,  $m+i)|  =  $m+i)  >  (l+o(l))gm||$m+i||i,  [0.79m1/6]. 

The  sequence  {«„  }  should  satisfy  the  relation 

Qm+l  =  (%mi  9m+ 1)  =  Tn+lll^C  m  ||2  +  —  {%rm  ^m+l) 

m 

and  by  (3.5)  and  (3.13) 

(3-14)  =  Am\lm+1\\xm\\2  -  qm+1)  < 

m 

(1  +  o(l))((l  -  a)mqmy1p{  1  -  /3)(1  -  2/3)~1m~1+/3, 

Let  us  specify  «o  :=  0.27  for  /3  =  1/6.  We  will  also  need  an  estimate  for  Am  —  Am_i. 
Denote 


(3.15)  9n+l  • —  9n+l  ^n+l^n+l  —  rJn-{~lXn  T  hn^- 1- 

Taking  into  account  that  coordinates  of  xn  are  negative  (see  (3.12))  and  coordinates 
of  hn+i  are  positive  and  less  in  absolute  value  than  the  corresponding  coordinates 
of  xn  (see  (3.9))  we  get  that  the  nonzero  coordinates  (g'n+1)k  of  g'n+i  are  negative 
and 

(3.16)  hn+lW  <  ln+l\\xn\\,  (dn+l)  k  <  ~{quln  ~  «0 /n),  k  >  an. 

We  have 


(3.17)  Am  —  (xm,  (Xm—  1,  $m)  — 

{Xm—  1)  ^m+1  ^m)  Qinidmi  ^m+l)- 

All  coordinates  of  xm-\  are  negative.  Therefore 

(3.18)  (1  T  o( l))c?m  —  A  (Xm.—l-t  1  4?m)  A 

T  1 1 | |oo  =  (1  T  o(l))Q,m  T  ||*^m||oo) 


where 


/ 

OO  * 


max  \am,k 

k>am 
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From  (2.7)  and  definition  of  {hn}  we  get 

k/a 

(3.19)  ll^mlToo  <  (1  +  o(l))qm(l  +  a0  max  V'/-1)^ 

“  “  l=k 

(1  +  o(l))qm{l  +  cko  In  1/a),  [1.51m-5/6]. 

Thus  we  get  from  (3.18)  for  (3  =  1/6  and  big  enough  N 

(3.20)  —0.84m-5/6  <  $m+i  —  $m)  <  0.68m-5/6. 

We  get  from  (3.16) 

(3.21)  |  ((/m,  |  7  IlSrnll  ||^m+l  ||  —  7m  \  \  %m—  1 1 1 1 1  'f’m+l 1 1  ~ 

(1-o)i/2(1-/3)(1-2/3)-‘/2,  [1], 

On  the  other  hand  by  (3.16)  we  get 

(3.22)  (, gm ,  $m+i)  <  -(1  -  a)m(qrn^rn  -  a0/m ),  [-0.53]. 

Thus  from  (3.17),  (3.20),  (3.21),  and  (3.22)  we  get  for  big  N 

(3.23)  —0.4m-5/6  <  Tm_x  -  Am  <  1.52m-5/6. 

Further, 

(3.24)  Aj  -  Aj  1  =  (Am-i  -  Tm)(Tm_iTm)-1. 

Using  (3.13)  we  get  for  /3  =  1/6  from  (3.24)  and  (3.23)  that 

(3.25)  —0.641m-7/6  <  Am]  -  Am1_1  <  2.43m-7/6. 

We  proceed  now  to  estimate  \(xn,gm+ 1)|.  Consider  first  the  case  m  >  n.  We  will 
prove  (3.1)  in  this  case  by  induction  using  the  following  representation  formula 
(2.19) 

(3.26)  (x„,9m+l)  =  7"1~'  q<"  un,fhn)  +  (xn,hm+ 1  -  2^±1  ftm). 

7m  q-m—l  7‘ in 

By  (3.7)  and  induction  assumption  we  get  for  the  first  term 

(3.27)  |7m+i  <  (1  -  (1  +  o(l))m-1)gn+i. 

7m  Qm— 1 

Thus  we  need  to  prove  that  there  exists  <5  >  0  such  that 

(3.28)  \{xn,hm+1  -  ————hrn')  |  <  (1  -  <S)m-V+i. 
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We  have 

(3.29)  (xn,  hm+ 1  -  = 

Tm 

(®7i)  (^m+1  ^m))  4“  \Xn,  &m.(  1  ))  — •  Ul  “1“  02- 

m  m  7m.  to  —  1 

We  have  by  (3.19) 

(3.30)  |ai|  =  \(xn,  ^-($m+i  -  $m)>  < 

m 

lla;n||/o0Q!m+1  <  (1  +  o(l))(l  +  CKO  In  l/a)gn+iQ;0m_1,  [0.41m-1n-5/6]. 

m 

For  a2  we  have 


(3.31) 


\a2\  <  \\xr 


1$, 


CXm+1 

m 


7m+ 1  CKm 
7m  m  ~  1 


Using  (3.14)  we  get 

(3.32)  “ — '  -  7m+1  C>mi  =  -  ^m1_l)(7m+ll|*ml|2  -  <Wl)  + 

m  7 m  m  —  1 


^■m— 1  ("Tm+l  ( |  |^m  1 1  ||®m— 1||  )  9m+l  4“  <?m)  — •  0"l  4"  0"2- 

7  m 

Taking  into  account  (3.5)  we  get  from  (3.25)  the  following  estimates  for  07 

(3.33)  —0.14m-2  <  cti  <  0.51m-2. 

Let  us  proceed  to  02  now.  We  have 

(3.34)  7m+i ( 1 1 xm 1 1 2  -  ||xm_i||2)  -  qm+i  +  ^^-qm  «  -/3(1  -  /3)m-1 

7 m 


Using  this,  (3.13)  and  the  following  inequality  for  Am 

Am  <  ||iCm||||^m+i||  ~  (1  -  /3)(1  -  2/3)-1/2(l  -  a)1/2™/3,  [m1/6], 

we  get 

(3.35)  — 0.18m-2m-2  <  cr2  <  —0.13m-2. 

Thus 

—0.32 m-2  <  (Ji  +  <72  <  0.38m-2. 

and 

(3.36)  |  a,2 1  <  |cri  4-  0"2 1 1 1  1 1 1 1  1 1  <  0.38m-3/2n- 3/3 . 

Combining  (3.30)  with  (3.36)  we  get 

(3.37)  |ai|  4-  | a,2 1  <  0.79 m-1n-5^6  <  0.99m-1gn+i. 
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This  inequality  holds  for  all  m  >  n  and  big  enough  N.  Using  (3.7)  we  inductively 
get  from  (3.26)  and  (3.37)  that  for  all  m  >  n 

(3.38) 

|  (xn,  gm+l)  |  <  Qn+1- 

We  proceed 

now  to  the  case  m  <  n.  By  (2.11)  and  (2.13)  we  need  to  estimate 

71 

1  Y  (h^9k)\,  k  =  N  +  1,.... 

l=k-\- 1 

We  use  the  definition  hi  =  Trr&i  and  the  inequality  0  <  cq  <  ckq  which  holds  if  N 

is  big  enough. 

If  l  >  20 k  then  ( hi,gk )  =  0.  For  l  =  k  +  1  we  have 

\{hi,gk)\  =  \{hi,g'k)\  <  IMUW  <  Ck -1. 

For  k  +  2  <  l  <  20 k  we  have 

(3.39) 

and  by  (3.16) 

|{4, , g'k)\  <A1/2||9;,||<L5. 

Thus 

\{hi,gt)\  <  0.27P1,  l>k+  2, 

and  therefore 

20  k 

(3.40) 

Y  \(hh9k)\  <  (1  +  o(l))0.271n20  <  0.81. 

l=k+ 1 

Thus  we  have  for  m  <  n 


(3.41)  \{xn,gm)\  <  (0.81  +  0.05)gn+i  <  qn+1 

for  e  =  0.05cpv+i  and  big  N. 

Let  us  complete  the  construction  of  a  counterexample.  Let  N  be  big  enough  to 
ensure  (3.38)  and  (3.41)  for  n,m>  N.  Then  we  set 

N 

Xn  ■=  —  JV-‘/=5(24)-V^-V»^ei 


i—  1 


and 


9N+1  :=  II^AT  II  2  +  £jV+le-V+l 

with  (pv+i  chosen  from  (3.3)  with  (3  =  1/6  and  £,n+i  >  0  such  that  ||^jv+i||  =  1-  We 
define  now  {xn}n>N+1  and  {gn}n>N+ 2  by  (2.2)  and  (2.4)  with  {yn},  {hn}  specified 
in  Section  3  with  (3  =  1/6.  We  set  e  =  0.05qN+i  and  define 

gN  :=  exN+i\\xN+i\\-2  +  ^gN+i,  Hfll/vll  =  1- 

Consider  now  /  :=  xn+i  and  V  :=  {gN,  gN+i,  ■  •  •  We  have  that  /  e  Span(gjv,  gN+i) 
and  therefore  |/|^.1(x>)  <  C(N).  By  (3.38)  and  (3.41)  the  PGA  with  regard  to  V 
applied  to  /  will  exactly  realize  the  iterative  process  (2.4).  Thus 

fm  —  %N+ 1+  m 

and 


ll/m||2  =  ^(iV  +  l  +  m) 


-2/3 


This  implies 


\\fm\\  >  C1(N)m^1^3 

which  completes  the  proof  of  Theorem  1.1. 
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4.  Proof  of  Theorem  1.3 

The  construction  of  Vt  is  the  same  as  in  Section  2  of  [LT],  It  uses  the  Equalizer 
procedure.  Let  H  be  a  Hilbert  space  with  an  orthonormal  basis  For  two 

elements  e*,  ej,  i  ^  j,  and  for  a  positive  number  t  <  1/3  we  define  the  procedure 
which  we  call  ’’equalizer”  and  denote  E(ei,ej,t). 

Equalizer  E(e*,  ej,t).  Denote  /o  :=  e*  and  g\  :=  coe*  —  (1  —  a\ )1//2ej  with  a\  :=  t. 
Then  ||#i||  =  1  and  (f0,gi)  =  t.  We  define  the  sequences  g2,...,gN] 

a2  ajv  inductively: 

fn  ■  fn  —  1  {fn—  li  9n)9m  9n+ 1  •  ^ra+lG  (1  ®ri+l)  ^ 
with  CKn+i  satisfying 

(fni  9n+ 1)  1,2,.... 

Let  fn  =  anei  +  bnej  and  N  :=  Nt  be  the  number  such  that 

o>N— i  —  6jv— i  ^  \/2 1,  cijv  —  ^iv  ^  y/2it. 

Then  we  modify  the  N-th.  step  as  follows.  We  take  gjy  :=  2 _1/2(ej  —  ej)  and 

/at  =  /jv-i  -  (fN-i,gN)gN- 
It  is  clear  that  then  a/v  =  fr/v  and 

t  <  (/at-1)  filv)  <  2b 

We  list  here  the  following  simple  relations 

®n+l  —  6n+l  —  bn  T  t(l  ,  Tl  <!  IV  1, 


(4.1)  un_i_i  6n_i_i  —  cin  bn  t(an+i  T  (1  ^  ),  ti  N  1, 

||/n+l||2  =  ||/n||2-t2,  n<N-l. 

Relation  (4.1)  and  the  inequality  1  <  a:  +  (1  —  a:2)1/2  <  21/2,  0  <  a:  <  1,  imply  that 

(4.2)  N  <l/t 
and 

ll/ivll2  >  II/at-iII2  -  4t2  >  ||/||2  -t-3t2. 

It  is  clear  that  E(ei,  ej,t )  is  a  WGA  with  regard  to  the  dictionary  e*,  gi,  g2,  ■  ■  ■ ,  <pv 
with  the  ’’weakness”  parameter  t. 

We  define  WGA  and  a  dictionary  as  follows.  We  begin  with  /  :=  e\  and 
apply  E(e i,  e2, t).  After  W  >  1  steps  we  get  g®, . . .  ,  g°N±  and 


/ 1  =  ci(ei  +  e2) 
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with  the  property 


II/1 1 12  >  ||/||2  —  (t  +  3f2)  =  \\f\\2(l  —  t  —  3t2). 

We  use  now  E(e  1,  e%,  t)  and  E(e 2,  e±,t).  After  2Nt  >  2  steps  we  obtain  g/, . . . ,  g\Ni 
and 

f2  =  c2(e  1  H - +  64) 

with  the  property 

||/2||2>||/1||2(l-t-3t2)>||/||2(l-t-3t2)2. 

After  s  iterations  we  get 

fs  =  cs(ei  +  •  •  •  +  e2s) 

and  apply  E(e{,  ei+2« ,  t),  *  =  1,2,...,  2s.  We  make  2s  W  >  2s  steps  and  get 
-  S^2-lVt  and 

/s+1  =  cs+i(ei  +  •  •  •  +  e2s+i) 

with  the  property 

iir+1n2  >  ii/n2(i-t-3t2)s+i. 

Thus  we  have 

(4.3)  ||/  -  Gl  (/,  Vt)  ||  >  (1  -  t  -  3 f2)s ,  S  =  l,2,..., 

with  the  dictionary 

Vt=  \J  ekU  (J  gf. 

fee  N  s>0;l<l<23  Nt 

The  relation  (4.3)  and  monotonicity  of  ||/  —  G^(/, Dt)||  prove  the  Theorem  1.3. 

Remark  4.1.  The  estimate  (1.5)  implies  that  for  small  t  the  parameter  a  in  (1.6) 
can  be  taken  close  to  1/4.  The  inequality  (4-3)  implies  that  the  parameter  b  in  (1.7) 
can  be  taken  close  to  (ln2)_1. 
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